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(S) Adaptive buffer/quantizer control for transfonn video coders. 

(57) A buffer/quantizer controller (110) in a video coder (100) utilizes the average quantization step size 
employed for the previous frame, the average number of bits produced per pel by the video coder for the 
Image contained in the previous fr^me and the current frame buffer (108) fullness level to obtain an 
indication of image complexity. A target current frame quantization step size to be employed by the 
video coder in encoding a current frame is based on the indication of image complexity. Optionally, the 
target frame quantization step size is adjusted by a mapping procedure, at predetemnined intervals 
within each frame period, to account for the cunrent buffer fullness. The map employed by the mapping 
procedure is adjusted in accordance with the target frame quantization step size. 
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Technical Field 

This invention is related to video image proces- 
sing and, more particularly, to adjusting the quanti- 
zation step size so as to regulate the degree of buffer 
fullness in a video coder. 

Background of the invention 

Transform coding techniques and systems for 
compressing video motion pictures are well-known. 
An output buffer and a buffer/quantizer controller are 
essential elements of this type of video coding sys- 
tem. 

Several methods for adjusting the quantization 
step sizes and thereby regulating the fullness level of 
the output buffer have been proposed. However, for 
certain image types these prior methods may requare 
drastic changes in the value of the quantization step 
size employed during the compression of a single 
image firame. This, in turn, results in non-uniform qual- 
ity when the resulting compressed Image contained 
within the frame Is reconstructed. Such non-uniformity 
in quality creates a displeasing effect that Is undesir- 
able to viewers of the reconstructed image. 

Other prior methods for regulating quantization 
step sizes and output buffer fullness explicitly conv 
pute statistical analyses of the images to be proces- 
sed. Such methods are capable of producing 
compressed images which can be reconstructed with 
nearly uniform quality. However, these prior methods 
require a large number of computations to perform the 
necessary analyses which are in addition to those 
computations needed to accomplish the regular cod- 
ing process. This requirement of added computations 
-increases the-cost of-such video coders. - - 

Another problem encountered in employing low 
bit rate video coding is that the channel rate can at 
times be insufficient to encode the image contained in 
each frame with good quality. Thus, a tradeoff may be 
had between the frame rate, i.e., the number of frames 
transmitted per second (frames/sec), and the quality 
of the images contained within those frames. Such a 
tradeoff nriay be fixed for an image sequence, i.e., a 
constant frame rate coding mode is employed, or it 
may be adaptively specified, i.e., variable frame rate 
coding mode is employed. A constant frame rate cod- 
ing mode, although simple to Implement, is inefficient 
because it is incapable of adjusting the coding of an 
image sequence to match the actual motion content 
an image complexity of the sequence. Typical prbr 
variable rate coders did not optimize the number of 
frames coded to achieve the t>est possible results 
within given parameters. 

Summary of the Invention 

The invention as defined in claims 1 and 11. 



Brief Description of the Drawing 

In the Drawing: 

A simplified block diagram of an exemplary trans- 
5 form video encoder is shown in FIG. 1; 

FIG. 2 illustrates the notation of time instances 
employed; 

Shown in FIG. 3, in simplified block diagram for- 
mat, is an exemplary buffer/quantizer controller; 
10 Shown in FIG. 4, in simplified block diagram for- 

mat, is an expanded view of target operating point 
estimator; 

FIG. 5 illustrates, in flow chart form, the operation 
of a target operating point estimator; and 
15 Shown in FIG. 6 is an exemplary mapping charac^ 

teristic employed by a quantization step size 
adjustment unit 

Detailed Description 

20 

A simplified block diagram of examplary trans- 
form video coder 100 is shown in FIG. 1. A digital 
Image sequence signal VIDEO IN 101 Is supplied as 
input to inter/intra transform coefficients generator 

25 102 which derives and supplies as an output signal 
103 comprised of blocks of transform coefficients of 
either original image values or interframe differences. 
Typically, a Discrete Cosine Transform (DCT) is used 
to transform spatial domain signals (image pels or 

30 frame differences) into frequency domain coefficients. 
Transform coefficient generators and the techniques 
employed therein, e.g., conditional replenishment or 
motion compensation, are well-known. 

The transform coefficients in signal 103, from 

3S inter/intra transfonn coefficients generator 102, are 
- ~ supplied to quantizer-1 04, wherein the coefficients are- 
selected and quantized in order to reduce the trans- 
mission bit rate. The value of the quantization step 
size employed by quantizer 104 controls the quality of 

40 images reconstructed firom the quantized transform 
coefficients, as well as, the number of bits needed for 
reconstruction. Additionally, the number of transform 
coefficients to be transmitted is typically controlled by 
quantizer 104, which employs a coefficient selection 

45 procedure that is regulated by the quantization step 
size. Such selection procedures are well known. 
Quantized coefficients 105 are supplied to entropy 
coder 106 wherein they are further compressed. Out- 
put bit stream 107, which is comprised of the com- 

50 pressed coefficients supplied by entropy coder 1 06, Is 
stored in output buffer 108. These compressed coef- 
ficients are supplied by output buffer 108 as signal 
compressed data out 109, which may be transmitted 
over a communication channel. Output buffer 108 is 

55 required because the number of bits generated by 
entropy coder 106 varies significantly at different 
areas of an image sequence. Output buffer 108 stores 
the bit stream comprised of the compressed coeffi- 
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cients temporarily, in a first-in, first out-fashion. 
Additionally, output buffer 1 08 regulates the output bit 
rate of signal compressed data out 109 so as to match 
a subsequent receiver of the signal, e.g., a channel. 

In order to generate the interframe differences, in 
either the spatial domain or in the frequency domain, 
quantized coefficients 105 are fed back to inter/intra 
transform coefficients generator 102. The operations 
of quantizer 1 04. entropy coder 1 06, and output buffer 
108 are synchronized in time and are well-known. 

FIG. 2 illustrates the notation of time instances 
employed. Two exemplary frames are shown and the 
time scale of each is noted. In general terms, the data 
of any image frame number n is processed between 
time t(n,1) to t(n*1,1) which represents one frame 
time period, buffer/quantizer controller 110 generates 
M adjustments of the quantization step size during the 
time from t(n,1) to t(n.M). In accordance with an 
aspect of the invention. Buffer/quantizer controller 
1 10 at the beginning of processing firame n, i.e., t(n,1), 
generates an estimated quantizer step size, 
q_EST(n), which, if employed for each time Interval of 
frame n, is predicted to result in output buffer 108 
being filled to a predetennined level desired by the 
implementor, B_DESIRED. Operation of quantizer 
104 such that at each time interval quantization step 
size q_EST(n) is actually employed and output buffer 
108 is filled to level B_DESIRED is defined to t>e the 
target operating point for frame n. Additionally, in 
accordance with an aspect of the invention, at each 
time instance t(n,j) between and including t(n,1) and 
t(n,M) a quantization step size that is actually used by 
quantizer 104 for that time interval is determined by 
buffer/quantizer controller 110. The derivation of the 
quantization step size actually employed from the 
target operating point is via a mapping process that 
will be described in more detail below. Although in this 
example and the those below M is fixed, this should 
not be construed as a limitation and it is not beyond 
the scope of this invention for M to vary from frame to 
frame. 

In accordance with an aspect of the invention, 
buffer/quantizer controller 110 (FIG. 1) is supplied by 
entropy coder 106 with the average number of bits per 
pel, b_AVG(n-1) 111, employed In frame n-1, i.e.. the 
previous frame. Additionally, the current buffer full- 
ness B(t(n,j)) at each time interval j is supplied as sig- 
nal B(t(n,j)) 1 1 2 by output buffer 1 08. The quantization 
step size for each time interval q(t(n-1,j)) is employed 
to derive an average quantization step size for frame 
n-1 , namely q_AVG(n-1). In accordance with another 
aspect of the invention buffer/quantizer controller 110 
generates for each interval t(n,j) a quantbiation step 
size q(t(n,j) 113 which is supplied as output to be 
utilized by quantizer 104. The interval M is chosen so 
that buffer/quantizer controller 110 adjusts the quan- 
tization step sizes frequently enough so that output 
buffer 108 does not overflow or underflow and the 



quantization step size is maintained neariy constant 
for the entire image contained within the firame. Main- 
taining the quantization step size neariy constant for 
the entire image results in an image quality that is 

5 neariy uniform throughout. 

Shown in FIG. 3, In simplified block diagram for- 
mat, is an exemplary buffer/quantizer controller 110 
(FIG. 1). Average quantization step size computation 
unit 301 is supplied with each quantizer step size and 

10 computes an average quantizatton step size for the 
prior frame n-1 . Each such average is computed prior 
to the start of processing of frame n. The average may 
be computed by storing previous values of the quan- 
tization step sizes or on a running basis. The average 

15 is reinitialized to zero and computed independently for 
each frame. 

Once each frame, attime t(n,1), the average num- 
ber of bits per pixel used to encode frame n-1, 
b_AVG(n-1) 111, the average quantization step size 

20 used to encode frame n-1 , q_AVG(n-1 ), and B(t(n, 1 )) 
112, the output buffer 108 level attime t(n,1) are sup- 
plied, In accordance with an aspect of the invention, 
to target operating point estimator 302. Target operat- 
ing point estimator 302 computes an estimate of the 

25 target operating point for quantizer 104 (FIG. 1) by 
determining target quantization step size q_EST(n) 
for the selected target buffer fullness level B_DES- 
IRED. Target quantization step size q_EST(n) is 
thereafter supplied to quantization step size adjust- 

30 ment unit 303. Operation of target operating point 
estimator 302 is described in more detail below. 
Additionally, in accordance with an aspect of the 
invention, quantization step size adjustment unit 303 
perfonms a mapping based on q_EST(n) from the sup- 

35 plied fullness level at t(n,j) of output buffer 108, 
B(t(n,j)) 1 1 2, to the actual quantization step size to be 
used between time t(n,j) and t(n,j+1), q(t(n,j)) 113. The 
nature of this mapping wilt be described further below. 
Shown in FIG. 4, in simplified block diagram for- 

40 mat, is an expanded view of target operating point 
estimator 302. As shown, target operating point 
estimator 302 is comprised of image characteristic 
parameter computation unit 401 and quantization 
step size estimator 402. In brief, image characteristic 

45 parameter computation unit 401 obtains an estimate 
of the expected complexity of the current image F(n). 
Thereafter, quantization step size estimator 402 
employs this estimate of image complexity and the 
current buffer fullness level to determine the esti- 

so mated quantization step size, q_EST(n). 

FIG. 5 illustrates, in flow chart form, the operation 
of target operating point estimator 302 (FIG. 3). The 
routine is entered via step 501 at the beginning of 
each frame. In step 502 the image characteristic par- 

55 ameter of frame n, F(n), is computed according to 
F(n)=[q_AVG(n-1)]2.e« »>-AVG(n-i)^ where a is a coder 

dependent parameter, e.g., 1.39. The image charac- 
teristic parameter of a frame is an indication of the 
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complexity of the image contained within the frame. 
Steps 501 and 502 are perfonmed by image charac- 
teristic parameter computation unit 401 (FIG. 4). 

Thereafter, a loop is begun in step 503 by initiali- 
zing T, the estimated time duration for processing a 
frame. The value of T is set to T_MIN, the minimum 
time that can be allotted for processing to encode a 
frame. Typically T_MIN is the time required to process 
frames containing the simplest images. The number 
of bits to be removed from output buffer 1 08 by virtue 
of having been transmitted to the channel is calcu- 
lated by b_REMOVED=C T in step 604. C is the chan- 
nel bit rate (bits/sec). C could be a fixed value if this 
encoder is designed to operate at only a single chan- 
nel rate, or C may be a value negotiated between the 
encoder and the channel (and possibly the decoder), 
if this encoder can operate at several bit rates. T_MIN 
Is selected by the Implementor, eg., T_MIN = 1/30 
sec, although such a choice may be restricted by 
hardware devices. 

The number of bits per pel available to encode 
frame n, namely b_EST(n), Is estimated in step 505 
by 

b EST(n) - P-PESIRED-> B(t(n,1)) + b_REMOVEP 
~ num_pels 

where B_DESIRED is the desired level for output buf- 
fer 108 (FIG. 1), which is typically half of the buffer 
size (B_MAX) and num_pels is the total number of 
pels in frame n, which is often a fixed value for a speci- 
fic coder. The estimated average quantization step 
size for frame n, which Is the target quantization step 
size q_EST(n), is thereaftr obtained in step 506 by 

_ --?-b ESTCn) 

<LEST(ii) = F(n)»^-c ^ . 

Conditional branch point 507 tests if q_EST(n) is less 
than the desired quatization step size q_DESIRED, 
which is selected by the implementor to maintain a 
reasonable picture quality, or if the T value is greater 
than T.MAX, the longest permissible duration be- 
tween two frames as selected by the implementor. If 
the test result in step 507 is NO. control is passed to 
step 508 wherein the value of T is increased by AT, a 
value selected by the implementor which is typically 
the duration of an input frame. Thereafter, control is 
passed to step 504 to repeat the loop. If the test result 
in step 507 is YES, control is passed to step 509 which 
exits the routine. q_EST(n) may then be supplied as 
output. Steps 503 through 509 are performed by 
quantization step size estimator 402 (FIG. 4) 

If the constant frame rate coding mode is desired, 
T_MIN can be set to the desired frame duration and 
T_MAX set equal to T_MIN. This will cause the loop 
comprised of steps 503 through 508 to be executed 
only once. Alternatively, the implementor can design 



a stopping condition test for step 507 based on the 
combination of both q_EST(n) and T. For example, 
one may choose to exit the loop when q_EST(n) is 
close to q.DESIREO and T is dose to T_MAX, 

5 although neither condition is met exactly. If variable 
frame rate coding is employed and if T is greater than 
the time necessary to process the current frame, at 
least one subsequent frame will be discarded. 
Methods of discarding subsequent frames and for 

10 assuring that a corresponding decoder does not 
become dysfunctional because of such discarding are 
well known. 

For processing the first input firame, n=1, 
q_EST(1) is chosen by the implementor, since 

IS b_AVG(0) and q_AVG(0) are not available. Typically, 
q_EST(1) is chosen appropriately to match the chan- 
nel bit rate C. For a large value of C, a small value may 
be chosen for q_EST(1). Also, for frame n=1. the 
implementor may desire to use a different mapping 

20 characteristic in quantization step size adjustment 
unit 303 (FIG. 3) discussed below. 

Shown in FIG. 6 is an exemplary mapping charac- 
teristic employed by quantization step size adjust- 
ment unit 303 (FIG. 3). The exact shape of this 

25 mapping characteristic (curve) is determined by the 
implementor. The target operating point of buf- 
fer/quantizer controller 110 is the point at which cur- 
rent estimated quantization step size q_EST(n) has 
the value expected to be employed for an entire frame 

30 time period so that the fullness level of output buffer 
1 08 (FIG. 1) is maintained at a predetermined desired 
level, B_DESIRED, selected by the implementor. 
Thus, the target operating point is defined by the point 
(q_EST(n), B_DESIRED) on the mapping characteri- 

36 Stic. Therefore, the preferred operating condition Is 

_ _ ,such.thatwhen_th9 level of output buffer 108. B(t(n,j)) 
1 1 2, is in the neighborhood of B_DESIRED, the quan- 
tization step size employed is approximately 
q_EST(n). Because of variations in the image, how- 

40 ever, it is desirable to decrease the quantization step 
size q(t(n,j)) 113, when B(t(n,j)) 112 is lower than 
B_LOW to avoid underflow in output buffer 108. Simi- 
lariy. when B(t(n,J)) 112 is higher than B_HIGH, it is 
desirable to increase the quantization step size 

45 q(t(n.j)) 113 to avoid buffer overflow. Therefore, at 
each of the M time intervals in a frame, the actual buf- 
fer fullness level of output buffer 1 08 Is checked. If the 
fullness level of output buffer 108 is greater than 
B_LOW and less than B^HIGH the quantization step 

50 size employed is q_EST(n), in accordance with FIG. 
6. Should the fullness level of output buffer 108 
exceed B_HIGH, the actual quantization step size 
employed is increased from the value of q_EST(n), in 
accordance with FIG. 6, up to the maximum quanti- 

55 zation step size possible, q_MAX. Likewise, in the 
event that the fullness level of output buffer 1 08 is less 
than B_LOW, the actual quantization step size 
employed is reduced firom the value of q_EST(n), in 
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accordance with FIG. 6, down to the minimum quan- 
tization step size possible, q_MIN. The exact shape of 
this mapping characteristic may also be constrained 
by the available quantization step sizes, e.g., only 
even numbers between and including 2 to 64 may be 
used in accordance with CCITT recommendation 
H.261 . For each value q_EST(n), a different mapping 
characteristic may be selected or one exemplary 
mapping characteristic may be modified in a predeter- 
mined manner prior to performing the mapping. 

Additionally, in order to prevent buffer overflow, 
quantizer 104 may have to select not to pass any of 
the coefficients to output buffer 108 when it is deter- 
mined that output buffer 108 is nearly full (close to 
B_IVIAX). Similarly, in the case of buffer underflow, it 
may be necessary for buffer/quantizer controller 110 
to send information to entropy coder 106 so that the 
latter can insert stuffing bits into the compressed bit 
stream. However, proper selection by the implemen- 
torof the mapping characteristic and the mechanisms 
for estimating the complexity of each image greatly 
reduces the probability of the above mentioned prob- 
lems arising. 



Clainns 

1. Apparatus [100] for dynamically determining a 
quantization step size for a video coder wherein 
said video coder is processing at least a portion 
of at least one video signal comprised of frames 
wherein each frame contains at least one image 
representation, comprising: 

quantizer means [104] for quantizing a 
digital signal representing said video signal com- 
prised of frames; 

means for obtaining an average quanti- 
zation step size employed by said quantizer 
means in a previous frame [301]; 

buffer means [108] for storing a quantized 
version of said digital signal representing said 
video signal comprised of frames; 

means for obtaining an indication of full- 
ness [112] from said buffer means [108]; and 



3. The apparatus as defined in daim 1 further 
including means for obtaining an average number 
of bits produced per pet by said video coder in a 
previous frame [111]. 

5 

4. The apparatus as defined in claim 3 wherein said 
means for generating is further responsive to said 
average number of bits produced per pel. 

10 5. The apparatus as defined in daim 1 further 
including means responsive to said estimated 
target quantization step size for generating at pre- 
determined intervals an adjusted quantization 
step size [303]. 

15 

6. The apparatus as defined in claim 5 wherein said 
means for generating said adjusted quantization 
step size [303] Includes means for mapping said 
estimated target quantization step size into said 

20 adjusted quantization step size. 

7. The apparatus as defined in claim 6 wherein said 
means for mapping Is responsive to said obtained 
buffer fullness level and a mapping characteristic 

25 to yield said adjusted quantization step size. 

8. The apparatus as defined in claim 6 wherein said 
means for mapping determines said mapping 
characteristic in a manner responsive to said esti- 

30 mated target quantization step size. 

9. The apparatus as defined in claim 8 wherein said 
means for mapping further includes means for 
selecting said mapping characteristic at predeter- 

35 mined intervals from a predetermined set of map- 

ping characteristics. 

10. The apparatus as deifned in claim 9 wherein said 
selection is based on said estimated target quan- 

40 tization step size. 

11. A method for dynamically determining a quanti- 
zation step size for a video coder wherein said 
video coder is processing at least a portion of at 
least one video signal comprised of frames whe- 
rein each frame contains an at least one image 
representation and said video coder includes at 
least quantizer means for quantizing a digital sig- 
nal representative of said video signal and buffer 
means for storing a quantized version of said digi- 
tal signal representative of said video signal, 
comprising the steps of: 

obtaining an average quantization step 
size employed by said quantizer means in a pre- 
vious frame; 

receiving an indication of fullness from 
said buffer means; and 

generating an estimated target quanti- 



means responsive to said average quanti- 45 
zation step size for said digital signal and said full- 
ness indication for generating an estimated target 
quantization step size [110]. 

2. The apparatus as defined in claim 1 wherein said so 
means for generating [110] includes means for 
deriving an initial estimated target quantization 
step size [302] and means for revising said initial 
estimated target quantization step size into said 
estimated target quantization step size which ref- 55 
lects an allotment of additional time for proces- 
sing a frame [402]. 



15 



35 
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zation step size, responsive to said average 
quantization step size and said fullness indi- 
cation, for said digital signal. 

12. The method as defined in claim 1 1 further includ- 
ing the step of generating at predetermined inter- 
vals an adjusted quantization step size 
responsive to said estimated target quantization 
step size. 

13. The method as defined in claim 12 wherein said 
step of generating is further responsive to an 
average numt>er of bits produced per pel. 
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FIG, 3 
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FIG. 5 
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FIG. 6 
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